System and method for motion compensation of image planes in color sequential displays

ABSTRACT

A system and method for motion compensation of color sequential displays, receiving a motion image data input, extracting an estimate of a motion of an image portion represented in said image data input with a motion estimator, and calculating, for a time instance of display of each image subframe of an image frame, a compensated representation for reducing display artifacts with a processor, wherein each subframe represents a different aspect of the image frame. The subframes are preferably different color planes of the image frame. The processor preferably performs a bilinear interpolation where the compensated image boundary does not fall on a display pixel boundary. The motion compensation algorithm is preferably robust in the face of erroneous motion vectors. The system and method is advantageously used in single panel liquid crystal display projection systems, although the techniques are applicable to various image display technologies.

FIELD OF THE INVENTION

The present invention relates to the field of color image displays, andmore particularly to color displays in which the color image planes areseparately updated or displayed. The invention also relates tocompensation of motion images for artifacts introduced by a discretetime representation thereof.

BACKGROUND OF THE INVENTION

Color image displays are of two general types. In a first type,exemplified by a typical direct view cathode ray tube color display, allcolor image components are displayed simultaneously. Thus, an imagemodel, e.g., a CCIR-601 signal, defines the luminance and chrominance ofeach image pixel at a particular time. The motion image is thereforepresented as a time sequence of color image frames.

In a second type of color image display, color image planes aredisplayed sequentially. This type of system is employed, for example, incertain single panel image projection systems, in which light of variouscolors sequentially illuminates a common spatial light modulator. Thespatial image modulator, therefore, modulates the intensity of eachrespective color component of a pixel sequentially and independently,which is perceived as a color motion image.

Sequential color displays work well as long as one pixel is beingobserved during an entire frame time. When a moving object is present,and the eyes are focussing on this object, and the eyes start trackingthe motion. The color breakup artifact that an observer notices at thatmoment is caused due to tracking of the motion by the eyes, i.e. theeyes follow a moving object by rotating the head and eyes while keepingthis object focussed on the same position on the retina.

Color sequential displays display the Red, Green, Blue (RGB) colorsalternating during a frame time, as represented in FIG. 1. This frametime might have a small delay for each successive row, depending on theway the color sequential illumination is implemented, but this timedelay is generally considered negligible. The image pixels are observedat different time moments within a frame period, and thus might displaydifferent video data for the RGB colors other than the intended one. Ifthis happens when the video data of these pixels changes, at that momenta color break-up artifact is visible.

JP 08-123355 A, published May 17, 1996, relates to a motion compensationsystem for a plasma display panel. In this system, the image motionartifact caused by the tracking of the human eye of motion of adisplayed object between successive frames, in a display system havingpulse modulated gray scale generation, representing a plurality of imagesubframes differing in brightness, is corrected by calculating objectmotion and moving the object within the image for correction at the timeof display.

The following references are hereby incorporated herein by reference intheir entirety:

-   W. Bruls, A. van der Werf, R. Kleihorst, T. Friedrich, E. Salomons    and F. Jorritsma, “A single-chip MPEG-2 encoder for consumer storage    applications” Digest of the ICCE, 1997, Chicago, pp. 262-263.-   G. de Haan and H. Huijgen, “Motion estimation for TV picture    enhancement”, in Signal Processing of HDTV III, (H. Yasuda and L.    Chiariglione, eds.), Elseviers Science Publishers B. V., 1992, pp.    241-248.-   G. de Haan, J. Kettenis, and B. Deloore, “IC for motion compensated    100 Hz TV, with a smooth motion movie-mode”, IEEE Transactions on    Consumer Electronics, vol. 42, May 1996, pp. 165-174.-   G. de Haan, P. W. A. C. Biezen and O. A. Ojo, “An Evolutionary    Architecture for Motion-Compensated 100 Hz Television,” in IEEE    Transactions on Circuits and Systems for Video Technology, Vol. 5,    No. 3, June 1995, pages 207-217.

“Motion—compensated picture signal interpolation”, U.S. Pat. No.5,495,300, Inventors: G. de Haan, P. Biezen, A. Ojo, and H. Huijgen.

-   G. de Haan, P. Biezen, H. Huijgen, and O. Ojo, “True motion    estimation with 3-D recursive search block-matching,” IEEE    Transactions on Circuits and Systems for Video Technology, vol. 3,    No. 5, October 1993, pp. 368-388.-   G. de Haan, P. W. A. C. Biezen, H. Huijgen and O. A. Ojo, “Graceful    Degradation in Motion-Compensated Field-Rate Conversion” in    Proceedings of the International Workshop on HDTV, Ottawa, Canada,    1993, pages 249-256.-   JP 06-46358, 18.2.1994, “Liquid Crystal Driving Device”, Masao    Kawamura. (Moving image emphasis system).-   WO 96/35294, “Motion Compensated Filtering”, 7 Nov. 1996, Timothy    Borer (Motion compensated filtering of interlace video). See also    U.S. Pat. Nos. 5,335,194; 4,862,266; 4,862,267.

SUMMARY OF THE INVENTION

The present inventors have therefore determined that the perceivedartifacts evident in a color sequential display producing an image of amoving object may be addressed by separately motion compensating theobjects represented within compensation scheme determines motion vectorsfor areas or objects within an image stream, predicts (e.g., byinterpolating or extrapolating) the object position for respective timesof presentation of respective color subframes, and sequentially displaysthe respective color planes representing the areas or objects at thepredicted position.

Artifacts may be evident any time a dynamic pixel image is displayed ata time other than the theoretical timing of presentation. Thus, whilethe artifact is especially apparent in color sequential RGB displaysdriven using the nominal RGB color separation of a sequence of compositeframes of a video signal, it may also occur in certain other instances,for example in systems employing other color plane definitions or colorspace representations. Therefore, it is understood that the invention isnot limited to the modification of color plane images of RGB sequentialcolor displays to correct for the non-coincidence of the actual time ofdisplay with the theoretical time of display with respect to the motionof objects within a frame. In fact, the invention encompasses themodification of image subframe data for display, especially where theimage subframes represent different components of the image frame andare presented at different respective times than the nominal frame time,by reformulating the discrete time image subframes by distinguishingimage object components having apparent independent motion, andresynthesizing an image subframe with a modified relationship of therespective object components having independent motion. Preferably, thisresynthesis is for the purpose of correcting a timing shift between theactual time of display of the image subframe including the objectcomponent and the theoretical time for display based on the originalscene. The present invention also provides a system and method forcalculating and presenting the image corrections with sub-pixelprecision.

The prediction of object or area position may be of any known type,although interpolation or extrapolation of first order (velocity) ispreferred. Higher order predictions, for example, require additionalinformation, and may therefore increase image presentation latencyand/or memory requirements. While known motion estimation techniques,such as those employed in MPEG-2 encoders, may be employed, model basedsystems may also be employed, for example MPEG-4 or VRML-type systems,to define the object motion.

Motion estimation systems typically operate on image frames to analyzesequential changes. Where analogous regions are identifiable betweensequential frames, typically confined to a search area, the displacementof the region may be encoded as a motion vector. Therefore, thisinformation is often used in motion image compression, wherein theidentification of an image block and its motion vector is transmitted,instead of the image block itself. According to the present invention,similar techniques may be used, as is well known in the art. However,portions of a moving foreground object may cover or uncover thebackground image. In the case of uncovering of background, thebackground image need be synthesized. In the case where an imageinterpolation is employed, the background image may be predicted basedon the subsequent image frame. In the case of an extrapolation (i.e.,use of past frame data only), this information may be lacking.Therefore, in spite of the use of intelligent algorithms, artifacts maybe generated in this case as well.

According to the present invention, it is important to encode the motionvectors to represent the movement of objects represented in the image,with priority given to the largest motion vectors, rather than thegreatest opportunity for image compression.

The implementation of a processor for computing motion vectors is wellknown in the art, and need not be described herein in detail. Typically,the motion vector compensation system will include a powerful generalpurpose computer or workstation, or a dedicated digital image processingdevice, such as an appropriately programmed Royal Philips ElectronicsTrimedia device.

Motion estimation analyzes the speed of the video frame content and themotion vectors are determined. Motion compensation, on the other hand,uses these motion vectors to re-align (or interpolate) the three RGBcolors according to these motion vectors. In FIG. 2, the motion artifactand the result of motion compensation are shown. These artifacts occurgenerally in sequential color display systems, although an analogousartifact exists in Plasma Display Panels (PDPs) and Digital MirrorDevices (DMDs).

According to the present invention, in color field compensation, thecolor fields are aligned on the motion vectors (with the remainingrounding errors). In color sequential displays, the colors are alignedon the motion vectors with a remaining rounding error, or moreaccurately with the preferred bilinear interpolation technique. This ispossible in color sequential displays, since the entire amplitude forone color is being displayed in one color subframe, in contrast to colorfield (PDP) gray scaling. According to an aspect of the invention, bitsplitting techniques, frame rate increasing techniques, and motioncompensation may be simultaneously employed, by controlling eachsubfield and each color plane individually. For example, when usingtraditional bit splitting of the most significant bit (MSB), when makinga correction of one-half of the MSB subfields, the other half MSBsubfield is also “falsely” corrected. According to the presentinvention, two bits are available for controlling each half of the MSBsubfield individually.

Therefore, it is an object according to the present invention to providea system and method for image compensation in an image display device inorder to avoid various motion image artifacts, especially image colorbreak-up and improve a perceived imaged quality.

It is a further object of the invention to provide a color sequentialimage display processor which reduces color break-up by estimating anoptimal position of an image representation of a moving object or areabased on a time of presentation and predicted or estimated motioncharacteristics.

It is also an object according to the present invention to provide asystem for motion compensation of color sequential displays.

Such a system comprises means for estimating motion from the input imagedata; means for calculating a motion compensated luminance value fromthe input video data, valid at a temporal instance that corresponds to agravity center of each color subframe; and means for calculating thevalue for all color subframes leading to correct perception of themotion for the combination of the color subframes. The calculation ofthe correct color subframe video value is preferably done by bilinearinterpolation; alternately, the calculation of the correct colorsubframe video value is done by selecting the closest pixel to choosethe video data. The motion compensation means preferably includes arobust interpolation method reducing the effect of erroneous motionvectors, the robust interpolation method being selected from one or moreof median filtering and order statistical filtering. Advantageously, adecision of a reference time for the most important color subframe,e.g., green, is taken at a moment in time for which no motioncompensation interpolation, or compensation over the shortest possibledistance from the original picture, is required.

It is also an object according to the present invention to provide amethod for motion compensation of color sequential displays.

Such a method comprises the steps of:

-   -   (a) processing received image data comprising frames (2), each        frame defining a plurality of subframes (4, R, G, B), each        subframe representing a different component of the image frame        (R, G, B) for display at different respective times (TR, TG, TB)        within a frame period;    -   (b) estimating a motion of an image portion represented in said        frames of image data input (3); and    -   (c) motion compensating the image portion based on the estimated        motion, with respect to a respective time instance of display of        at least one of said subframes thereof (5), thereby reducing        display artifacts.

These and other objects and features of the present invention willbecome more fully apparent from the following description and appendedclaims taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, reference is made to thedrawings, which are to be taken in conjunction with the detailedspecification to follow:

FIG. 1 shows a subframe division for a color sequential device;

FIGS. 2A and 2B shows the nature of the motion artifact and motioncompensation thereof, respectively;

FIG. 3 shows the misalignment of the RGB video data to obtain motioncompensation;

FIGS. 4A and 4B show, respectively, a subframe reconstruction byrounding and by interpolation for a color sequential device;

FIG. 5 shows bilinear interpolation of the sub-frame value SF from theneighboring pixels at an image position; and

FIG. 6 shows a flowchart of a method according to the present invention;and

FIG. 7 shows a block diagram of a system according to the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention will now be described by way of the drawings, in whichcorresponding reference numerals indicate corresponding structures inthe figures.

As shown in FIG. 2, a white (R+G+B) vertical bar 6 pixels wide is movingover the screen over a black background with a speed of 3 pixels perframe. The observer's eyes track the motion according to the speed anddirection of the motion vectors. At the top of the motion vectors, thecolors have been shown that are observed when tracking this motion. Atthe leading edge of the line, a blue and mix of blue and green pixel isbeing observed, while at the trailing edge a mix of green and red, and ared pixel can be seen. Motion compensation attempts to project the RGBdata from one pixel on the motion vector. The result is that the samevertical bar is also shown in FIG. 2, and in this case only the whitebar is seen.

The observed luminance is the sum of the observed color subframes, SF,thus, $\begin{matrix}{{L(k)} = \sum\limits_{SF}} & (1)\end{matrix}$and is in this case equal to:L({overscore (k)})=R({overscore (k)})+B({overscore (k)})+G({overscore(k)})  (2)

For a moving object with speed, v=({overscore (v)}v_(x), v_(y)), that isbeing tracked by the observers eyes, the contributions of RGB are:R({overscore (k)})=r({overscore (k)}+d _(R)(({overscore (k)}))G({overscore (k)})=g({overscore (k)}+d _(G)({overscore (k)}))B({overscore (k)})=r({overscore (k)}+d _(R)({overscore (k)}))  (3)with R({overscore (k)}), G({overscore (k)}), B({overscore (k)}) beingthe RGB colors that are observed simultaneously on one position in theeye, r({overscore (k)}), g({overscore (k)}), b({overscore (k)}) beingthe input RGB video data, {overscore (k)}=(x, y) the position on thescreen and {overscore (d)}d_(R), {overscore (d)}_(G), {overscore(d)}_(B), the distance over which the RGB colors for an observer seem tobe displaced, according to Equation 4:{overscore (d)} _(R) ={overscore (v)}·( t _(R) −t ₀){overscore (d)} _(G) ={overscore (v)}·( t _(G) −t ₀){overscore (d)} _(B) ={overscore (v)}·( t _(B) −t ₀)  (4)with {overscore (v)}, the motion vector at position {overscore (k)}, t₀the reference time within a frame and t_(R),t_(G),t_(B), the emissiontime of the R, G and B colors.

These equations only represent the displacement of an object withvelocity v during a time t_(R)−t₀ (for red)

For a static object in a scene i.e., the horizontal speed v_(x)=0 andvertical speed, v_(y)=0, this results in L({overscore (k)})=r({overscore(k)})+b({overscore (k)})+g({overscore (k)}), which exactly equals theinput RGB video data. When the speed is not equal to 0, the observedluminance is the sum of the RGB data, but shifted by the motion vectorand the time difference between the emission moment of a color and thereference time.

The observed mis-convergence in the colors during motion tracking is thedistance over which the video data seems to be shifted according to theEquations 4.

This mis-convergence relative to a position on the screen k can beexpressed as:{overscore (k)}+{overscore (d)} _(R){overscore (k)}+{overscore (d)} _(G){overscore (k)}+{overscore (d)} _(B)for red, green, and for blue respectively, and are the positions on thescreen that are being observed during motion tracking and the emissionmoments of the RGB colors.

If the reference time, t₀, is taken equal to the moment of the emissionof green, t_(G), the mis-convergence can be calculated, and results in:{overscore (k)}+{overscore (d)}_(R) for red, {overscore (k)} for greenand {overscore (k)}+{overscore (d)}_(B) for blue.

To be precise, when a position {overscore (k)} on the screen is beingobserved, only red and blue have a mis-convergence of {overscore(d)}_(R) and {overscore (d)}_(B), which is exactly the displacement ofthe eyes over the screen due to the speed {overscore (v)} and theemission time moments of red and blue respectively, relative to green.It is convenient to take green as a reference, since in that case thereis no mis-convergence for low speeds, furthermore, an error in themotion vector only leads to a smaller error in the colors red and blue.The human eye is less sensitive for a positional error in blue, which isa reason not to choose blue as a reference time. The time differencebetween the reference time and the colors red and blue is smaller thanwhen the reference time is chosen to be equal to the start of a frame.Errors in the motion vectors lead in that case to a largermis-convergence, especially for blue.

For compensation of the mis-convergence of the colors, it is attemptedto make the observed positions on the display, under tracking of anobject (in FIG. 2, the directions of the motion vectors), equal for allthree RGB colors. Thus, by mis-aligning of the video data for the RGBcolors with respect to each other, motion compensation can be obtained.

The mis-alignment for red, {overscore (p_(R))}, green, {overscore(p_(G))}, and blue, {overscore (p_(B))} at position {overscore (k)} onthe screen, to compensate for this error can be calculated according to:{overscore (k)}+{overscore (d)} _(R) +{overscore (p _(R) )} {overscore(=k)}+{overscore (d)} _(G) +{overscore (p _(G) )}= {overscore(k)}+{overscore (d)} _(B)+i {overscore (p_(B))}  (6)If the moment of emission of green equals the reference time, this leadsto:{overscore (k)}+{overscore (d)} _(R)+{overscore (p _(R) )}= {overscore(k)}={overscore (k)}+{overscore (d)} _(B)+{overscore (p _(B) )}  (7)

Thus, to perceive a good motion compensated image during tracking, themis-alignment of red and green for an object with velocity {overscore(v)} must be:{overscore (p _(R) )}=− {overscore (v)}·( t _(R) −t _(G)){overscore (p _(B) )}=−{overscore (v)}·( t _(B) −t _(G))

In FIG. 3, the displacement of red and blue has been indicated, and themis-alignment that is required to obtain motion compensation for a speedof 2 pixels per frame.

FIG. 3 also illustrates a particular problem which is seen in a matrixdisplay panel. The video data for red and blue must be displayed on thepanel at a position in between two pixels, i.e. for red it is a positionof ⅔rd from x and ⅓rd from x-1, and for blue it is at ⅔rd from x and ⅓rdfrom x+1. This, however, is a position at which no actual pixel boundaryis present, and therefore it is not possible to show the edge at exactlyat this location.

Two solutions are available. A first solution is to round themisalignment value of red, {overscore (p_(R))} and blue, {overscore(p_(R))} to the nearby pixels at a position {overscore (k)} on thescreen.

In that case Equation 8 becomes:{overscore (k)}+Round( {overscore (p _(R) )})= {overscore (k)}−Round({overscore (v)}·(t _(R) −t _(G))){overscore (k)}+Round( {overscore (p _(B) )})= {overscore (k)}−Round({overscore (v)}·(t _(B) −t _(G)))  (9)withRound({overscore (x)})=Round( x, y)=(Round(x), Round(y))  (10)This condition is shown in FIG. 4A.

In this case, an alignment error of at most ½ pixel remains in thecolors red and blue, which is generally considered acceptable.

The other solution is to use bilinear interpolation to reconstruct thevideo data for one color according to FIG. 4B.

The video data for the color subframe SF that is determined isinterpolated in such a way that after the mis-alignment, the video datatransition occurs exactly on the matrix grid.

First, the rounded value is determined according to Equation 9, afterthat the fractions {overscore (f_(R))} and {overscore (f_(B))} aredetermined for the bilinear interpolator for red and blue respectively,according to Equation 11.{overscore (f _(R) )}=Round( {overscore (p _(R) )})− {overscore (p _(R))}=Round( {overscore (v)}·( t _(R) −t _(G)))−{overscore (v)}·(t _(R) −t_(G)){overscore (f _(B) )}=Round( {overscore (p _(B) )})− {overscore (p _(B))}=Round( {overscore (v)}(t _(B) −t _(G)))−{overscore (v)}·(t _(B) −t_(G))  (11)

In FIG. 5, a representation of the bilinear interpolation is shown.

In general, the bilinear color subframe interpolation for a colorsubframe SF can be written as: $\begin{matrix}{{a = {{SF}\left( \overset{\rightarrow}{k} \right)}}{b = {{SF}\left( {\overset{\rightarrow}{k} + \begin{pmatrix}1 \\0\end{pmatrix}} \right)}}{c = {{SF}\left( {\overset{\rightarrow}{k} + \begin{pmatrix}0 \\1\end{pmatrix}} \right)}}{d = {{SF}\left( {\overset{\rightarrow}{k} + \begin{pmatrix}1 \\0\end{pmatrix} + \begin{pmatrix}0 \\1\end{pmatrix}} \right)}}} & (12)\end{matrix}$The value of ${SF}\left( {\overset{\rightarrow}{k} + \begin{pmatrix}f_{x} \\f_{y}\end{pmatrix}} \right)$is calculated as:SF({overscore (k)})=(1−f _(x))((1−f _(y))a+f _(y) c)+f _(x)((1−f_(y))b+f _(y) d)  (13)with f_(x) and f_(y) being the positive sub-pixel fraction in horizontaland vertical direction, respectively, resulting from motion.

Thus, the new video data for color subframe SF is determined by:SF({overscore (k)}+Round({overscore (p _(SF) )}))= D _(SF)({overscore(k)}+{overscore (f _(SF))})  (14)whereby the new video data of color subframe SF is mis-aligned byRound({overscore (p_(SF))}) (new position on the screen), and isinterpolated from the original color subframe video data D_(SF). Thecolor subframe SF can be either red or blue.

This bilinear interpolation is thus similar to typical antialiasingschemes, which provides a partial brightness distribution between pixelssurrounding the theoretical location in order to avoid edge artifacts.Therefore, it is also understood that other techniques known in thefield of image antialiasing may be applied to this system. In fact,according to the present invention, edges or edges of motion compensatedobjects may be processed to enhance the presented image, for example byincreasing contrast, in order to improve a perceived image quality andreduce artifacts.

The Motion Estimator

Rather than calculating all possible candidate motion vectors, therecursive search block-matcher takes spatial and/or temporal “predictionvectors” from a 3-D neighborhood, and a single updated predictionvector. This implicitly assumes spatial and/or temporal consistency. Theupdating process involves update vectors added to either of the spatialprediction vectors.

Assuming blocks of height Y, width X, and center {right arrow over (X)},we define a candidate set CS({right arrow over (X)},t), from which, attime t, the block-matcher selects its result vector: $\begin{matrix}{{{CS}\left( {\overset{\rightarrow}{X},t} \right)} = \begin{Bmatrix}\left( {{\overset{\rightarrow}{D}\left( {{\overset{\rightarrow}{X} - \begin{pmatrix}X \\Y\end{pmatrix}},t} \right)} + {{\overset{\rightarrow}{U}}_{1}\left( {\overset{\rightarrow}{X},t} \right)}} \right) \\\left( {{\overset{\rightarrow}{D}\left( {{\overset{\rightarrow}{X} - \begin{pmatrix}{- X} \\Y\end{pmatrix}},t} \right)} + {{\overset{\rightarrow}{U}}_{2}\left( {\overset{\rightarrow}{X},t} \right)}} \right) \\\left( {\overset{\rightarrow}{D}\left( {{\overset{\rightarrow}{X} - \begin{pmatrix}0 \\{2Y}\end{pmatrix}},{t - T}} \right)} \right)\end{Bmatrix}} & (15)\end{matrix}$where T is the picture period, and the update vectors {right arrow over(U)}₁({right arrow over (X)},t) and {right arrow over (U)}₂({right arrowover (X)},t) are block-alternatingly equal to the zero vector ({rightarrow over (0)}), or taken from a limited fixed integer update set, inour case: $\begin{matrix}{{{US}_{i}\left( {\overset{\rightarrow}{X},t} \right)} = \begin{Bmatrix}\overset{\rightarrow}{0} & \quad & \quad & \quad \\{\overset{\rightarrow}{u}}_{y} & {- {\overset{\rightarrow}{u}}_{y}} & {- {\overset{\rightarrow}{u}}_{x}} & {- {\overset{\rightarrow}{u}}_{x}} \\{2{\overset{\rightarrow}{u}}_{y}} & {{- 2}{\overset{\rightarrow}{u}}_{y}} & {3{\overset{\rightarrow}{u}}_{x}} & {{- 3}{\overset{\rightarrow}{u}}_{x}}\end{Bmatrix}} & (16)\end{matrix}$where we introduce ${\overset{\rightarrow}{u}}_{x} = {{\begin{pmatrix}1 \\0\end{pmatrix}\quad{and}\quad{\overset{\rightarrow}{u}}_{y}} = {\begin{pmatrix}0 \\1\end{pmatrix}.}}$

To realize sub-pixel accuracy, the update set of equation (9) isextended with fractional update values. An overall quarter pictureelement (pel) resolution is achieved by adding the following fractionalupdate vectors to the update set: $\begin{matrix}{{{US}_{i}\left( {\overset{\rightarrow}{X},t} \right)} = \begin{Bmatrix}{\frac{1}{4}{\overset{\rightarrow}{u}}_{y}} & {{- \frac{1}{4}}{\overset{\rightarrow}{u}}_{y}} & {\frac{1}{4}{\overset{\rightarrow}{u}}_{x}} & {{- \frac{1}{4}}{\overset{\rightarrow}{u}}_{x}}\end{Bmatrix}} & (17)\end{matrix}$

The estimator chooses its output motion vector {right arrow over(D)}({right arrow over (X)},t) from the candidates, using the meanabsolute difference (MAD) criterion. Because of the small number ofcandidate vectors that have to be evaluated, the method is veryefficient, i.e. only a few mean absolute differences have to becalculated. Furthermore, due to the inherent smoothness constraint, ityields very coherent vector fields that closely correspond to thetrue-motion of objects.

The Motion Compensated Interpolation

Motion compensation can be very straightforward, i.e. merely fetch theluminance from a position shifted over the estimated motion vector.Although simple, such a simple method shows rather strong artifacts incase of erroneous vectors. Such vector errors cannot always beprevented, as some temporal effects in an image sequence cannotadequately be described in terms of translations. Therefore, a robustmotion compensation algorithm should preferably be applied. Robust hereis meant in a sense that the algorithm includes protection mechanismsthat prevent extreme degradations in the event of erroneous motionvectors.

To that end, rather than Dust shitting, or slightly better, averagingthe motion compensated luminance values from the neighboring pictures:F _(a)({right arrow over (x)}, t−α)=½(αF({right arrow over (x)}+α{rightarrow over (D)}, t), +(1−α)F({right arrow over (x)}−(1−α){right arrowover (D)}, t−T)); 0≦α≦T  (18)which is a more common procedure, a robust algorithm according to thepresent invention performs a non-linear filtering of motion compensatedand non-motion compensated pixels:F _(mc)({right arrow over (x)},t−α)=Med{F( {right arrow over(x)}+α{right arrow over (D)}, t), F({right arrow over (x)}−(1−α){rightarrow over (D)}, t−T), F _(av)({right arrow over (x)}, t)}  (19)where F_(av) as defined as: $\begin{matrix}{{F_{av}\left( {\overset{\rightarrow}{x},t} \right)} = {\frac{1}{2} \cdot \left( {{F\left( {\overset{\rightarrow}{x},t} \right)} + {F\left( {\overset{\rightarrow}{x},{t - T}} \right)}} \right)}} & (20)\end{matrix}$and Med is the median function, defined as: $\begin{matrix}{{{Med}\left( {A,B,C} \right)} = \left\{ \begin{matrix}{A,} & {\left( {B < A < C} \right)\bigvee\left( {C < A < B} \right)} \\{B,} & {\left( {A \leq B \leq C} \right)\bigvee\left( {C \leq B \leq A} \right)} \\{C,} & {otherwise}\end{matrix} \right.} & (21)\end{matrix}$

Of course, it is understood that other known robust algorithms forfiltering a predicted boundary position and/or centroid position may beemployed.

The image processor according to the present invention may beimplemented with a highly integrated processor, such as the PhilipsElectronics TriMedia processor, which, for example, may provide otherfunctions, such as digital video decoding (e.g., MPEG-2), audio decodingand processing, and the like. The digitized image signal frames arereceived in yuv color space and converted to RGB color space. As shownin FIG. 6, a set of successive frames are received 2, and the motion ofimage objects within between image frames is estimated 3. Generally,regions with the fastest moving objects require compensation more thanregions with slowly moving objects, so that the image is, for example,processed in order of magnitude of computed motion vector, (or densityof image change between successive frames) within the availableprocessing time. The RGB color subframes are then defined at an instantin time 4 motion compensated 5 based on the respective time of display,and output 6 in known manner.

As shown in FIG. 7, YUV image data is input 10 into both a Yuv processor11, which, for example, performs image scaling, and a motion processoror motion estimator 14, which analyzes the YUV image data forsignificant image regions with common motion vectors. The processed YUVimage data is then converted to a pixel representation in RGB colorspace in YUV to RGB converter 12. The RGB image in the form of RGBsubframe information is then processed in RGB processor 13, whichperforms, for example, gamma correction, histogram adaption, etc. Theoutput processor or color motion compensator 15 receives the motioncompensation information in the form of processed RGB date from themotion processor or motion estimator 14, and the separated, processedRGB subframe information from the RGB processor 13, and compensates theimage of each respective RGB subframe, based on a prospective time ofpresentation, to reduce the perceived motion artifact, by means ofmotion compensated RGB sequential data 16.

The output processor 15 may also provide other functions, for exampleapparent frame rate increasing, e.g., by inserting a greater number ofcolor subframes within a frame period than the minimum three for an RGBrepresentation, each of which is preferably compensated for itsrespective time of presentation.

In practice, all of these functional elements may be implemented assoftware-defined constructs within a dedicated processor, such as theRoyal Philips Electronics TriMedia processor.

While the above detailed description has shown, described and pointedout the fundamental novel features of the invention as applied tovarious embodiments, it will be understood that various omissions andsubstitutions and changes in the form and details of the system andmethod illustrated may be made by those skilled in the art, withoutdeparting from the spirit of the invention. Consequently, the full scopeof the invention should be ascertained by the appended claims.

1-15. (canceled)
 16. A system for motion compensation of displayspresenting a plurality of image subframes within an image frame period,each subframe representing a different component of the image frame,comprising: (a) a motion estimator for estimating motion from the inputimage data; (b) a processor for calculating a motion compensatedluminance value from the input image data, valid at a temporal instancethat corresponds to a presentation of each respective image subframe;and (c) means for calculating a pixel image value for all image subframecomponents leading to correct perception of the motion for thecombination of the image subframe components.
 17. The system accordingto claim 16, wherein each image subframe corresponds to a color plane ofan image frame.
 18. The system according to claim 16, wherein thecalculation of the pixel image value for an image subframe component isdone by bilinear interpolation.
 19. The system according to claim 16,wherein the calculation of the pixel image value for an image subframeis done by selecting the closest output pixel to the motion compensatedluminance value.
 20. The system according to claim 16, wherein theprocessor executes a robust interpolation algorithm for reducing theeffect of erroneous motion vectors, said robust interpolation algorithmbeing selected from one or more of median filtering and orderstatistical filtering.